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ABSTRACT 

A fieldwork procedure that enables students to design 
and conduct meaningful evaluation research in one semester is 
described. The procedure has been used in a 13-week program 
evaluation course in a master's program in community psychology and 
industrial/organizational psychology. About 2 months before the 
course begins, administrative units within the university are invited 
to subnit requests for evaluation assistance. At the first class 
meeting, a list of potential evaluation projects are distributed to 
students. In most cases teams of two or three students get their 
first choice of projects. Before the third class meeting, student 
teams are to meet with their project sponsors to start negotiating 
the specifics of their evaluations. By the fifth class meetf q, teams 
submit written evaluation plans to the course instructor and their 
Pi.oject sponsors. Projects share at least four characteristics that 
have contributed to their success: they are client-driven, 
small-scale, regulated by strict deadlines, and campus-based. 
Outcomes for 10 sponsors who worked with student teams over 2 years 
are briefly discussed. (SW) 
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Field Experiences in Evaluation Courses: 
Increasing Their Value to Students and Sponsors 

Previous authors have expressed doubts that students can 
design and conduct a meaningful piace of evaluation research 
in one semester. This paper describes and analyzes a fieldwork 
procedure that attempts to provide students with such an 
opportunity. Consumer satisfaction data from both students 
and project sponsors suggest that the procedure has been successful. 
Projects share at least four characteristics that have contributed 
to their success: They are client-driven, small-scale, regulated 
by strict deadlines, and canpus-based. Limitations of the one- 
sonester approach are discussed. 
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Field Experiences in Evaluation Courses: 
Increasing Their Value to Students and Sponsors 

Incorporating meaningful field experiences into Program Evaluation 
(PE) courses is a continuing diallenge for instructors (e.g., Brwn 1985? 
Conner, 1986? Weeks, 1982; Wortiran, Cordray, & Reis, 1980)' The appealing 
scenario of a two-sennester PE seminar acccnpanied by a year-long PE intern- 
ship ii: simply not a viable option in many training programs for a variety 
of logistical reasons. A more typical situation is one in which a one-semes- 
ter PE course involves a field-oriented project of sane sort (e.g., Conner, 
1986) . For an instructor comnitted to providing studerts with an opportunity 
to design and conduct an evaluation from stare to finish, the latter schedule 
inposes serious constraints. Moreover, producing evaluations that will actually be 
used by decision-makers — no easy feat under even the best of circumstances — 
becomes an especially intimidating task ^fA\en the time frame encorpasses just 
a single semester. 

It is the more frequently encountered scenario that is the subject of 
this paper. What can an instructor do to maximize the quality of the student's 
hands-on learning experience in the one-semester course? And v^at are the 
iirplications of this analysis for hew the clients of student evaluators in 
short-term projects might be better served? The framework I will use for 
addressing these issues is the canpus-based field training component I have 
integrated into the master's- level PE course I regularly teach at a 
small, urban university. 
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Background and Structure of the Course 
Most students v*io take PE at my institution are enrolled in a master's 
program in either canmnity psychology or industrial/organizational psy- 
chology and have catpleted a graduate- level intrcxJuctory course in research 
methods. Both full-and part-time students take the 13-week PE course^ 
which meets once a week for three hours. Fran 12 to 15 students are usually 
enrolled. 

Pre-Course Preparation 

^proximately two months before the course begins I send a menorandu^n 
to a wide variety of administrative units within the laniversity (e.g., academic 
departments, the alumni office, the counseling center) , inviting them to sub- 
mit requests for evaluation assistance. Since I cannot assume that all 
recipients of the memo are familiar with evaluation research, a brief descrip- 
tion of the various types of evaluation is included. 

I screen all requests for their general acceptability and make follow- 
up calls in response to all written submissions. (Seme individuals simply 
I*ione in their requests.) Projects that would priJfiarily consist of having 
students analyze pre-existing sets of data are usually judged to be unaccept- 
able. In most cases these data sets have been "lying around" the deoart- 
ment or division for sane time with no action having been taken, a fairly 
reliable predictior of 1cm ccmmitment to the evaluation project on the part 
of the decision-makers involved. In addition, such projects do not provide 
students with an opportunity to develop and iitplement a data-collection 
strategy, an iitportant evaluation task. 
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Most requests are for needs assessments or process evaluations, 
with a few asking for evaluations of program outcomes. Thus 
far there have been no proposals for cost-benefit or cost-effectiveness 
analyses. In view of the fact that I do not discuss cost-oriented evalu- 
ations until late in the PE course, a lack of requests in this area is 
probably all to tlie good. 
1st Class Meeting 

I distribute a list of potential evaluation projects to all students. 
In addition to the cairpus-based projects, there are typically one or two 
off-campus projects included on the list. These latter proposals are from 
settings that I have had contact with in a professional or volunteer capacity, 
and are generated through a much more informal process than the on-canpus 
requests. 

Before the 2nd Class Meeting 

Students form teams of 2 or 3 and submit their project preferences to 
me. To avoid placing an undue burden on project sponsors, I ask students 
to refrain from contacting them during this period. Instead, I try to 
provide students with sufficiently detailed information in my written and 
oral project descriptions to enable than to make enlightened choices. 

Elnployed students vrtio do not wish to participate in one of the listed 
projects can satisfy the field requirement by doing a program evaluation 
v*iere they work. Prior to the second class they must submit to roe a brief 
description of what they plan to do. The criteria for judging the accepta- 
bility of these prx^sals are the same as those used for other projects. 
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2nd Class Meeting 

Project assignments are announced. In nost cases student teams get 
their first choice, and in no instance has a team ever been assigned to 
a project that was lower than its second choice. 
i3efore the 3rd Class Meeting 

Student teams are expected to meet with their project sponsors to begin 
negotiating the specifics of their evaluations. Because the teams only 
have one week in which to schedule and hold these meetings, I encourage spon- 
sors to be as acccranodating as possible when first contacted by the team. 
3rcl Class Meet ing 

Teams share with the rest of the class the results of their initial meet- 
ings with project sponsors. Problems and issues are brainstormed, and various 
action strategies are discussed. 
By the 5th Class Meeting 

Teams submit written evaluation plans to me and their project sponsors. 
These plans discuss in detail the focus and purpose of the evaluation, as 
well as the procedures to be followed in carrying it out (including a time- 
table) . The resources required for the project are set forth, along with 
an analysis of any ethical issues raised by the project. 

If the evaluation plan is submitted late the grade received by the 
team suffers significantly. This policy motivates teams to confront 
rather than avoid problems when designing the evaluation. Because of the 
shortness of a 13-week semester, procrastination in developing a plan can 
easily have fatal implications for the plan's implementation. Stringent 
time frames are a reality frequently encountered by practicing evaluators. 
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and students ?re almost certainly better off if they learn this fact 
sooner rather than later. 
By the 12th Class Meeting 

Teams submit final evaluation reports to me and the project sponsors. 
The format for the report can vary widely depending on the nature of the 
project. Once again^ a significant grade penalty is assigned if the report 
is late. 

Before the 13th Class Meeting 

Teams are expected to meet with sponsors to review and discuss the 
final report. These meetings occasionally lead to one or two follow-up 
sessions after the course is over. 
13th Class Meeting 

Teams share with the rest of the class the results of their wrap-up 
meetings with project sponsors. The overall structure of the fieldwork 
coTiponent of the course is discussed and critiqued by the class as a vAiole, 
and reccnmendations for change are solicited. And the end of a very stren- 
uous semester is celebrated with food and drink provided by both myself and 
the students. 

Outcomes 

Surveys of project sponsors after the close of the semester indicate 
that the perceived usefulness of the evaluations is very high. Indeed, 
all 10 sponsors who worked with student teams over the past two years 
of this course gave the highest rating possible (5) in answering the question: 
"Overall, hew useful to you was the evaluation project?" (l=Tiot useful at 
all, 5=ver>' useful). Moreover, when sponsors were asked to estimate the im- 
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portapce of the decisions that ware actually affected by the evaluation 
an average rating of 4.4 out of a possible 5 resulted (1 = not inportant 
at all, 5 = very iirportant) . 

High levels of sponsor satisfaction are also suggested by the signi- 
ficant growth in requests for evaluation assistance over the years, fron 
an initial level of 5 or 6 per course to the current level of approximately 
15. Given the relatively small size of the university in which the course 
is taught, a large part of this inci"ease is undoubtedly due to word-of- 
mouth advertising that is positive in tone. 

The overv^elming majority of students also appear to view the evaluation 
project in a very positive li^t, despite the very heavy workload v^ich this 
corponent of the course iitposes on them. For exaitple, over the past two 
years they have responded with an average rating of 4.6 to the question: 
"Hew much do you feel you learned from doing the evaluation project? 
(1 = very little, 5 = a great deal, N = 28) . Moreover, during the last 
class meeting of the semester students typically corment that they "couldn't 
imagine" the course being offered without the fieldwork corponent. 

The high degree of satisfaction that most students display toward the 
project is prctoably largely due to the fact that the assignment embodies 
the characteristics of effective jdb design in a more general sense. That 
is, the tasks involved in the project are at least r.oderately high on the 
core dimensions of skill variety, autonomy, feedback, task identity, and 
task significance (Hackman & Oldham, 1980) . 
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Discussion 

The nost inportant conclusion suggested by irry experiences with one- 
semester evaluation projects is that careful (bordering on corpulsive) 
planning on the part of the instructor can generate short-term field 
experiences that expose students to all the major stages of a program 
evaluation v^ile influencing sponsors on iirporcant issues. Past dis- 
cussions of evaluation field training have not usually been very sanguine 
about the possibility of achieving these objectives within such a brief 
time frame (e.g.^ Conner^ 1986; Weeks^ 1982; Wortman et al., 1980), 

More specifically, there are four characteristics of the approach 
I have used which appear to have contributed to its success thus far: 
Projects are client-driven, small-scale, regulated by strict deadlines, 
and use or-canpus settings. The significance of each of these aspects 
will be briefly discussed. 
Client-Driven Projects 

Potential clients are carefully screened by myself and student teams 
to ensure that the requested evaluations will generate information that has 
a high probability of actually being used by decision-makers. The overall 
approach taken closely resembles Patton's (1986) model of utilization- 
focused evaluation. For exanple, a great deal of attention in the second 
class session is devoted to the topic of incorporating a utilization-focused 
perspective into negotiating and contracting with sponsors. In this context 
it is important to renember that, except for the initial letter announcing 
the PE course, there is no attempt made on my part to actively recruit project 
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sponsors. This reduces the probability of make-wrk projects being 
submitted. Overall, then, the design of the course's fieldwork ccmpo- 
nent maxiniizes the likelihood that the projects undercaken will involve 
sponsors of at least moderately high motivation vrtio are engaging in 
evaluation for thie "right" reasons. 
Small-Scale Projects 

Proposed evaluations that do not have a reasonable chance of being 
ccrtpleted within the 13-week semester are rarely accepted. Student teams 
are cautioned to keep the semester's length very much in wind when nego- 
tiating with sponsors concerning the various corponents of their projects. 

Sttiall-scale evaluations need not, and should not, be superficial eval- 
uations. Well designed, meaningful projects such as needs assessments 
and process evalutions can clearly be conducted in many campus 
settings in a semester. There are even opportunities to do certain types of 
outccme-oriented investigations (e.g., studies of the iirpact of remedial or 
probationary programs, evaluations of consumer satisfaction and decision- 
making) . Generally speaking, I believe that students ~ at least at the 
master's level — gain more from fully carrying out an evaluation of limited 
scope than from being involved in a subset of the phases associated with more 
ambitious projects. 
Strict Adherence to Deadlines 

The importance of this dimension has already been mentioned. While a 
realistic timetable is a key feature of virtually any successful evaluation, 
its role in a short-term project conducted by students is especially crucial. 
For one thing, th^re is sinply very little time available to regroup after 
missing a deadline, with the result being that subsequent deadlines are often 
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missed in donino-like fashion. And if the semester happens to end before 
the project is carpi eted, a variety of logistical factors typically come 
into play vrtiich can make it very difficult to finish the evaluation in a 
hi^-quality manner. Consequently, beginning with the very first class 
meeting I socialize students to take deadlines and tinietables very seriously. 
CaiTpus-Based Projects 

Emphasizing on-canpus rather than off-carrpus evaluations has several 
major advantages. The pre-course phase of developing projects is generally 
less time-consuming for the instructor witli on-canpus projects. In almost 
every case I am at least mcxSerately familiar with the department or office 
that requests assistance, in terms of both its activities and its formal 
and informal structure. This greatly expedites the prxx:ess of evaluating 
the quality of proposed projects, and allows me to hancie most issues over 
the phone. 

I can also be more confident that, vrtien I cormunicate to sponsors my 
expectations of them during the project, it will have more of an inpact 
on their behavior than would be the case with most off-canpus sponsors. 
This is because I frequently have direct or indirect working relationships 
with on-canpus sponsors in a variety of contexts, and these relationships 
could be severely jeopardized if either of us failed to follow through on 
our prcnises concerning the evaluation. This influence is especially bene- 
ficial when it results in sponsors taking the evaluation-oriented requests of 
student teams more seriously than they otherwise might. In this fashion it 
increases the likelihood that the ^^ery tight timetables for the project will 
actually be met. 
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Canpus-based projects facilitate the meeting of deadlines in another 
2S well. It is usually easier for student teams to handle the logistics 
of arr?.nging meetings with on-cairpus sponsors than off-canpus ones, in 
part due lo the greater flexibility inherent in the work schedules of the 
former. Given the fact that many meetings between sponsors and teams are 
usually needed over the course of the project, the advantages of on-campus 
projects in this respect can have major consequences for the success of the 
evaluation. 

The positive aspects of one-semester projects, whether they be on- or 
off-canpus, should not cause us to overlook their limitations. For e:"'^mple, 
most outcome-oriented evaluations, as well as those involving cost-benefit 
or cost-effectivenes-^ analysis, are simply not feasible within such a short 
time frame. This constraint contributes to a second limitation of one- 
senester evaluations: The methodologies they employ are usually not very 
hi^-pcwered in terms of their design or quantitative sophistication. 
Randomization and regression analysis are the stuff of course readings 
and classroom discussion, not fieldwork appl illation in most cases. At 
another level, the campus-based nature of most projects restricts the types 
of issues that can be addressed by the evaluations. Many of the concerns 
typically encountered in evaluations of human services programs, for example, 
are unlikely to be faced by student teams in their on-campus projects. 

These limitations, important as they are, appear to be more than 
compensated for by the benefits associated with one-semester, on-canpus 
projects. Having the responsibility for carrying out an evaluation from 
start to finish is an extremely valuable learning experience, especially 
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for master' s-level students who may be receiving no further graduate 
training in research methods or statistics. Consequently, instructors 
vrtio teach in programs where year-long evaluation internships are not a 
realistic option have much to gain from giving serious consideration to the 
model of field training presented in this paper and hew it might be adapted 
to fit their needs. 



o 1 4 



Field Experiences 
14 

References 

Brovn, R.D. (1985) . Supervising evaluation practicum students: A develop- 
mental nodel, Educationcil Evaluation and Policy Analysis ,> 2/ 161-167. 

Conner, R.F. (1986). The teaching of evaluation in interdisciplinary 
programs: UC Irvine. In B.C. Davis (Ed.), Teaching of evaluation 
across the disciplines (pp. 61-69) . San Francisco: Jossey-Bass. 

Hackman, J.R. & Oldham, G.R. (1980) . Work redesign , Reading, MA: 
Addison-Wes 1 ey . 

Patton, M.Q. (1986). Utilization-focused evaluation (2nd ed.). 

Beverly Hills, CA: Sage. 
Weeks, E.G. (1982). The value of experiential approaches to evaluation 

training. Evaluation and Program Planning ^, _5, 21-30. 
Wortman, P.M., Cordray, D.3., & Reis, J. (1980). Training for 

evaluation research: Sane issues. In L. Sechrest (Ed.), 

Training program evaluators (pp. 19-37). San Francisco: 

Jossey-Bass. 



15 



